The Open Systems Interconnection (OSI) Reference Model defines seven network protocol layers (L1-L7) used to communicate over a transmission medium. The upper layers (L4-L7) represent end-to-end communications and the lower layers (L1-L3) represent local communications.
Networking application aware systems need to process, filter and switch a range of L3 to L7 network protocol layers, for example, L7 network protocol layers such as, HyperText Transfer Protocol (HTTP) and Simple Mail Transfer Protocol (SMTP), and L4 network protocol layers such as Transmission Control Protocol (TCP). In addition to processing the network protocol layers, the networking application aware systems need to simultaneously secure these protocols with access and content based security through L4-L7 network protocol layers including Firewall, Virtual Private Network (VPN), Secure Sockets Layer (SSL), Intrusion Detection System (IDS), Internet Protocol Security (IPSec), Anti-Virus (AV) and Anti-Spam functionality at wire-speed.
Network processors are available for high-throughput L2 and L3 network protocol processing, that is, performing packet processing to forward packets at wire-speed. Typically, a general purpose processor is used to process L4-L7 network protocols that require more intelligent processing. For example, the Transmission Control Protocol (TCP)—an L4 network protocol requires several compute intensive tasks including computing a checksum over the entire payload in the packet, management of TCP segment buffers, and maintaining multiple timers at all times on a per connection basis. Although a general purpose processor can perform the compute intensive tasks, it does not provide sufficient performance to process the data so that it can be forwarded at wire-speed.
Furthermore, content aware applications that examine the content of packets require searching for expressions, which contain both fixed strings and character classes repeated a variable number of times, in a data stream. Several search algorithms are used to perform this task in software. One such algorithm is the Deterministic Finite Automata (DFA). There are limitations when using the DFA search algorithm, such as, exponential growth of graph size and false matches in a data stream with repeated patterns.
Due to these limitations, content processing applications require a significant amount of post processing of the results generated by pattern search. Post processing requires qualifying the matched pattern with other connection state information such as type of connection, and certain values in a protocol header included in the packet. It also requires certain other types of compute intensive qualifications, for example, a pattern match is valid only if it is within a certain position range within data stream, or if it is followed by another pattern and within certain range from the previous pattern or after/at a specific offset from the previous pattern. For example, regular expression matching combines different operators and single characters allowing complex expressions to be constructed.